For multiple scale factors of face super-resolution, a face super-resolution method based on very deep convolutional neural network was proposed; and through experiments, it was found that the increase of network depth can effectively improve the accuracy of face reconstruction. Firstly, a network that consists of 20 convolution layers were designed to learn an end-to-end mapping between the low-resolution images and the high-resolution images, and many small filters were cascaded to extract more textural information. Secondly, a residual-learning method was introduced to solve the problem of detail information loss caused by increasing depth. In addition, the low-resolution face images with multiple scale factors were merged to one training set to enable the network to achieve the face super resolution with multiple scale factors. The results on the CASPEAL test dataset show that the proposed method based on this very deep convolutional neural network has 2.7 dB increasement in Peak Signal-to-Noise Ratio (PSNR), and 2% increasement in structural similarity compared to the Bicubic based face reconstruction method. Compared with the SRCNN method, there is also a greater improvement. as well as a greater improvement in accuracy and visual improvement. It means that deeper network structures can achieve better results in reconstruction.